In the automotive safety field, the driver assistance systems can include a visual aid thanks to stereoscopic systems on-board vehicles. These systems are generally used for detecting obstacles located in the visual field upstream of these vehicles. A stereoscopic system indeed makes it possible to determine the distance between the vehicle and these upstream obstacles from two on-board cameras, arranged close to each other to provide stereo image pairs to a digital processing unit. By analyzing the disparity between the images provided in this manner, the system makes it possible to accurately identify the obstacles and the distance thereof to the vehicle.
The recognition of these obstacles is, moreover, notified to the driver by the driver assistance system. The reliability of the cameras can become decisive, for example when it is a matter of knowing in real time if, in the absence of obstacles signaled moreover, the road is definitely free of obstacles.
The accuracy is dependent on the calibration of the cameras and the knowledge of possible variations with respect to an initial calibration. The calibration of the cameras relates to the intrinsic parameters, such as setting the focal length thereof or the zoom thereof, and to the extrinsic parameters thereof relating to the position of each camera with respect to the vehicle and to the relative position of the cameras, with respect to each other.
Each camera is initially calibrated intrinsically in the factory and, for the supplier, the intrinsic parameters are considered to be constant for the entire duration of use.
Conventionally, since one of the cameras is considered to be the reference camera, the extrinsic calibration consists in setting the position and the rotation of this reference camera with respect to the vehicle and with respect to the rotation of the other camera, called the slave camera. The position of the cameras must be set very precisely with respect to each other to prevent any error of perpendicularity of the position thereof with respect to the spacing thereof. Yet, it is difficult to assemble them precisely enough to obtain a yaw zero offset and thus prevent this error.
Furthermore, the extrinsic parameters vary over time due to the variations in the parameters of use, in particular the variations due to the temperature or to the mechanical vibrations endured by the cameras.
With reference to an orthogonal coordinate system OXYZ of a stereoscopic system, the calibration of the relative rotation of the cameras about the transverse axis OX (pitch angle), the longitudinal axis OZ (roll angle) and the elevation axis OY (yaw angle) can advantageously be carried out by applying the epipolar constraint used in the search for the stereoscopic matching of the points in a so-called epipolar geometry space. This geometry establishes the relationships between the points of various images of a same scene (image points), produced on the basis of different viewpoints, these image points corresponding to the projections in the image space of the cameras of a same object point of the scene.
The epipolar constraint makes it possible to limit the search, in a given image, for the image point of an object point on a projection line called an epipolar line, while only the position of the image point in the other image is known. The epipolar constraint thus guides the construction of a stereoscopic image via the search for the matching points between each point of a mono acquisition first image, produced by a first camera, and the points of the epipolar line of the other image, produced simultaneously by the other camera. Epipolar geometry makes it possible to deduce, by simple relationships, the corresponding image points in conjunction with the depth of field thereof in order to reconstruct stereoscopic images, i.e. in three-dimensional vision.
However, the pixilation of the images has an impact on the quality thereof. This impact can be measured for the roll or pitch calibration since the detection of the calibration error can then be directly measured on the image. For example, a roll or pitch rotation of a degree will cause a deviation of 10 pixels on the image and this will be visible. However, the yaw deviation cannot be corrected on the image since the sensitivity is too low: the projection deviation on the epipolar line remains too small with respect to the image noise—less than 0.1 pixel on average for a shift of one degree—and the epipolar constraint cannot then be utilized.
To rectify this problem, and more generally to overcome the camera precise assembly error—which is particularly manifested on the yaw calibration—it could be envisaged to use additional external information, such as the vehicle speed or the depth of the scene from another sensor.
For example, the use of a radar allows an object—for example a vehicle—to be located at a given distance. The same vehicle is then observed with a first camera of the stereo system and the angle thereof calibrated with the other camera such that the vehicle is indeed at the given distance. However, the radar is not precise enough and therefore needs to take a large number of coordinate system points. Moreover, this radar generates an additional cost.
Other developments have been explored without the constraint of using a radar, using only the image processing system.
Thus, the patent document FR 2986358 describes the calibration of a camera mounted on a vehicle based on capturing specific target points located on a test pattern. By solving a system of nonlinear equations with six unknowns, three translational components and three rotational components, a point of coordinates given in the camera image plane is then positioned in the scene.
This solution is difficult to apply to the yaw calibration between two cameras of a stereoscopic system since the complexity of the system does not make it possible to produce unambiguous determinations for two moving cameras from a same test pattern.
The international patent application WO 2011/079258 proposes determining the real-time miscalibration of a multi-camera system, more particularly the extrinsic miscalibration thereof, and re-calibrating it, from the correspondence between observed data of an image—by the measurements thereof—and those provided according to the application of the calibration previously set. The correspondence of the data, which relates to features of typical object models, is stored as historical statistics of the alignment scores measured in real time.
However, this solution relates to systems with at least three multiple cameras and uses several multiple model objects, such as circles, or a 3D (three-dimensional) model, such as a cylinder, to implement the method. The use of standard models restricts the use of this method. In addition, the performances in determining the miscalibration, in particular the yaw miscalibration for an on-board stereoscopic system, are not measurable.